Add Cloud Snapshots: Named Save/Load/List of Emulator State (#335 P1) by thzgajendra · Pull Request #338 · stackshy/cloudemu

thzgajendra · 2026-08-07T11:58:32Z

Objective / Issue

First P1 item of the "minikube for cloud resources" roadmap (#335). Persistence (#337) auto-saves a single state on stop; this adds named, restorable snapshots on a live server — a local, free equivalent of LocalStack's paid Cloud Pods, built on the same persist engine.

What we found

The engine to capture/restore state already exists (persist.Export/Restore), and the admin control plane already does atomic whole-emulator rebuilds for /_cloudemu/reset. So snapshots are mostly a thin layer: an admin endpoint to move state in/out of the running daemon, plus a CLI to name and store it.
Blast radius: additive. New admin endpoint + new CLI subcommand; the persist-on-stop path is refactored to reuse a shared core but behaves identically.

How we fixed it / How it works

persist: optional Meta header on Snapshot + whole-emulator ExportAll/RestoreAll helpers (serve's persist-on-stop path now reuses them).
admin: GET /_cloudemu/snapshot (export whole-emulator state as JSON) and POST /_cloudemu/snapshot (rebuild-empty + restore). Both act on every provider like reset; wired via NewControl (nil = disabled, e.g. --admin=false).
CLI: cloudemu snapshot save|load|list|delete <name>. save/load talk to the running daemon; list/delete are file ops. Snapshots are single JSON files under ~/.cloudemu/snapshots/.

sequenceDiagram
    actor U as You
    participant CLI as cloudemu snapshot
    participant S as serve (daemon)
    participant P as persist
    participant F as snapshot file (disk)

    U->>CLI: snapshot save baseline
    CLI->>S: GET /_cloudemu/snapshot
    S->>P: ExportAll(all providers)
    P-->>S: Snapshot JSON
    S-->>CLI: state
    CLI->>F: atomic write (+meta header)

    Note over U,S: … a destructive test wipes/changes state …

    U->>CLI: snapshot load baseline
    CLI->>F: read
    CLI->>S: POST /_cloudemu/snapshot
    S->>S: rebuild to empty (reset)
    S->>P: RestoreAll(snapshot → providers)
    S-->>U: state restored (no restart)

Alternatives not taken

Stop-edit-file-restart — clunky and loses the "live" benefit; the admin endpoint restores in place.
A separate snapshot format — reused persist.Snapshot so snapshots auto-cover every service persistence gains, with zero snapshot-code change.

Docs / Tests / Playground

Docs: docs/standalone-server.md — named-snapshots section (uses, file location, --admin/provider requirement, name rules).
Unit: persist multi-provider ExportAll/RestoreAll; admin snapshot GET/POST/disabled(501); CLI name validation, flag parsing, and save→list→load→delete against a test server + daemon-down error.

Test plan

go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/...
Full local CI: gofmt / build / vet / go test ./... / go mod tidy / golangci-lint = clean (CodeQL: only pre-existing findings, none in changed files).
Live E2E via cloudemu start + real aws CLI on the default ~/.cloudemu: create bucket/object + DynamoDB table/item → snapshot save good-state → POST /_cloudemu/reset + create junk → snapshot load good-state → junk gone, all resources restored (bodies + items intact, aws+azure+gcp captured). Error paths: load-missing → not found; save ../evil → name rejected (traversal blocked); save while stopped → clear "not running".

Risk & Rollback

Low: additive. load wipes-then-restores (same semantics as reset) — intentional. Snapshot endpoint is gated on --admin; name sanitization blocks path traversal; writes are atomic.

Conclusion

cloudemu snapshot save/load/list/delete gives a free, local, multi-cloud, diffable "Cloud Pods": save a baseline, run destructive tests, restore in seconds, or hand a snapshot file to a teammate.

Follow-ups (#335): coverage grows automatically as more services join persistence; later — cross-machine sharing/registry, snapshot diff.

…tackshy#335 P1) Persistence (stackshy#337) auto-saves one state on stop; this adds multiple named, restorable snapshots on a live server — a local, free equivalent of LocalStack Cloud Pods, built on the same persist engine. - persist: add optional Meta header to Snapshot and whole-emulator ExportAll / RestoreAll helpers (serve's persist-on-stop path refactored to reuse them). - admin: GET /_cloudemu/snapshot exports the whole-emulator state as JSON, POST rebuilds to empty and restores from the posted state; both act on every provider like reset. Wired through NewControl (snapshot/restore may be nil to disable, e.g. --admin=false). - CLI: `cloudemu snapshot save|load|list|delete <name>`. save/load talk to the running daemon's control plane; list/delete are file operations. Snapshots are single JSON files under ~/.cloudemu/snapshots/, atomic-written, with name sanitization to block path traversal and --force to overwrite. Covers the same services as persistence (object storage, NoSQL tables, secrets, compute). Tests: persist multi-provider ExportAll/RestoreAll; admin snapshot GET/POST/disabled; CLI name validation, flag parsing, and save/load/list/delete against a test server.

NitinKumar004

Deep review — named cloud snapshots

Clean, correct, and well-tested. Verified in an isolated worktree at the PR head (#337 merged since, so the diff is the 10-file set). Gates green: build (darwin) + GOOS=windows go build ./cmd/cloudemu (the non-unix runSnapshot stub compiles) / vet / go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/ (ok all) / gofmt clean.

What's right:

Whole-emulator via one port done properly — snapshotFn/restoreFn close over all targets (like reset) and are wired through NewControl; nil closures → 501, so --admin=false disables it.
Restore semantics correct — validates SchemaVersion, rebuild()s to empty, then RestoreAll.
Secure by construction — name regex ^[A-Za-z0-9._-]{1,64}$ + explicit ./.. exclusion blocks path traversal (tested with ../etc, a/b, …); snapshot writes reuse #337's atomic temp+rename; POST body capped at 512 MiB.
Lock convention followed — the "read targets under rebuildMu, run Export/Restore outside it" pattern matches the existing seedFor, so no new concurrency regression.
Tests actually exercise the flows: name validation, save→list→load→delete against a test server, daemon-down, admin-off 501, multi-provider ExportAll/RestoreAll.

These are all Low / hardening / nit — nothing blocking. Two are inline; the third:

L1 (Low, hardening). The non-loopback --admin warning in serve.go currently says only that POST /_cloudemu/reset wipes state. This PR adds GET /_cloudemu/snapshot, an unauthenticated dump of all emulated state including secret values — a new network-reachable read/exfiltration surface on --host 0.0.0.0. Emulated data, so Low, but the warning should be broadened to cover read/exfiltration, not just wipe.

NitinKumar004 · 2026-08-07T12:17:10Z

+			return fmt.Errorf("%w: got %d, want %d", errUnsupportedSnapshot, snap.SchemaVersion, persist.SchemaVersion)
+		}
+
+		rebuild() // wipe to empty before loading


L2 (Low) — destructive load with no rollback. rebuild() wipes all state, then RestoreAll repopulates. If RestoreAll errors partway (a driver returns an error mid-restore), the emulator is left wiped + partially restored with no recovery of the prior state — the POST returns 400 but the data's already gone. Low probability (restore into freshly-empty providers rarely fails) and consistent with the documented reset-semantics, but worth a doc note now and, later, a "restore into a staging build and swap on success" hardening so a failed load can't destroy the running state.

Documented. Added a code comment on the rebuild() line and a note in docs/standalone-server.md making the destructive semantics explicit: load wipes then repopulates, and a mid-RestoreAll failure leaves the running state cleared. Called out your suggested hardening — restore into a staging build and swap in only on success — as the follow-up. 0f5c5a9

NitinKumar004 · 2026-08-07T12:17:10Z

+
+		fmt.Printf("%-24s %-20s %10d\n",
+			strings.TrimSuffix(e.Name(), ".json"),
+			info.ModTime().UTC().Format(time.DateTime),


L3 (nit). The CREATED column prints the file's ModTime, not the Meta.CreatedAt (RFC3339) actually written into the snapshot — and list ignores the stored Meta entirely, so it never surfaces the captured Providers. Reading the Meta header here would make the column accurate and let list show providers too.

Fixed. list now reads each snapshot's Meta header and shows the recorded createdAt plus a new PROVIDERS column, falling back to the file mtime / filename only when a file is unreadable or has no meta. Verified live: v1 2026-08-07T12:20:39Z aws,gcp 305. 0f5c5a9

…structive load Address review on the cloud-snapshots PR: - Broaden the non-loopback --admin warning: GET /_cloudemu/snapshot dumps all emulated state (including secret values) to any caller, not just reset wiping. - `snapshot list` now reads each file's Meta header, showing the recorded createdAt and a providers column instead of the file mtime alone. - Document that `load` is destructive (wipe then restore) and note the future staging-and-swap hardening for a failed restore.

thzgajendra · 2026-08-07T12:21:53Z

Thanks for the review — all three items addressed in 0f5c5a9 (all non-blocking, fixed anyway).

L1 (admin warning): broadened the non-loopback --admin warning to state that GET /_cloudemu/snapshot dumps all emulated state — including secret values — to any caller, not just that reset wipes state.
L2 (destructive load): documented the wipe-then-restore semantics (doc note + code comment); a mid-restore failure clears running state. Flagged your staging-build-and-swap-on-success idea as the follow-up hardening.
L3 (list Meta): list now surfaces the stored Meta — accurate createdAt and a PROVIDERS column.

Gate green: build / gofmt / go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/... / golangci-lint --new-from-rev = 0.

NitinKumar004

Approving — review comments addressed

Verified the hardening commit (0f5c5a9c) in an isolated worktree at the PR head; gate green (build darwin + GOOS=windows go build ./cmd/cloudemu / vet / go test -race ./persist/... ./server/admin/... ./cmd/cloudemu/ ok all / gofmt clean).

L1 — the non-loopback --admin warning now names GET /_cloudemu/snapshot dumping all emulated state (including secret values), not just reset wiping.
L2 — load's destructive wipe-then-restore is now documented in both the restoreFn comment and the docs, with the staging-and-swap future hardening called out.
L3 — snapshot list reads each file's Meta header (recorded createdAt + a providers column), with a clean filename/mtime fallback when Meta is absent.

One non-blocking nit the L3 change introduced: readSnapshotMeta json.Unmarshals each file into a full persist.Snapshot, so list allocates every object and base64-decodes all bodies (snapshots can be up to 512 MiB) just to read the small Meta header. Decoding into a minimal struct{ Meta *persist.Meta } would let json skip the big providers field — a cheap win whenever you touch this next. Not a blocker.

Clean, responsive work across the whole #335 snapshot line. LGTM.

NitinKumar004 reviewed Aug 7, 2026

View reviewed changes

NitinKumar004 approved these changes Aug 7, 2026

View reviewed changes

thzgajendra merged commit f0eaa88 into stackshy:development Aug 7, 2026
12 checks passed

Navigation Menu

Uh oh!

Add Cloud Snapshots: Named Save/Load/List of Emulator State (#335 P1) - #338

Add Cloud Snapshots: Named Save/Load/List of Emulator State (#335 P1)#338
thzgajendra merged 2 commits into
stackshy:developmentfrom
thzgajendra:feat/cloud-snapshots

thzgajendra commented Aug 7, 2026 •

edited

Loading

Uh oh!

NitinKumar004 left a comment

Uh oh!

NitinKumar004 Aug 7, 2026

Uh oh!

thzgajendra Aug 7, 2026

Uh oh!

NitinKumar004 Aug 7, 2026

Uh oh!

thzgajendra Aug 7, 2026

Uh oh!

thzgajendra commented Aug 7, 2026

Uh oh!

NitinKumar004 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

thzgajendra commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Objective / Issue

What we found

How we fixed it / How it works

Alternatives not taken

Docs / Tests / Playground

Test plan

Risk & Rollback

Conclusion

Uh oh!

NitinKumar004 left a comment

Choose a reason for hiding this comment

Deep review — named cloud snapshots

Uh oh!

NitinKumar004 Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

thzgajendra Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

NitinKumar004 Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

thzgajendra Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

thzgajendra commented Aug 7, 2026

Uh oh!

NitinKumar004 left a comment

Choose a reason for hiding this comment

Approving — review comments addressed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

thzgajendra commented Aug 7, 2026 •

edited

Loading